The augmented value of using clinical notes in semi-automated surveillance of deep surgical site infections after colorectal surgery.

BACKGROUND: In patients who underwent colorectal surgery, an existing semi-automated surveillance algorithm based on structured data achieves high sensitivity in detecting deep surgical site infections (SSI), however, generates a significant number of false positives. The inclusion of unstructured, clinical narratives to the algorithm may decrease the number of patients requiring manual chart review. The aim of this study was to investigate the performance of this semi-automated surveillance algorithm augmented with a natural language processing (NLP) component to improve positive predictive value (PPV) and thus workload reduction (WR). METHODS: Retrospective, observational cohort study in patients who underwent colorectal surgery from January 1, 2015, through September 30, 2020. NLP was used to detect keyword counts in clinical notes. Several NLP-algorithms were developed with different count input types and classifiers, and added as component to the original semi-automated algorithm. Traditional manual surveillance was compared with the NLP-augmented surveillance algorithms and sensitivity, specificity, PPV and WR were calculated. RESULTS: From the NLP-augmented models, the decision tree models with discretized counts or binary counts had the best performance (sensitivity 95.1% (95%CI 83.5-99.4%), WR 60.9%) and improved PPV and WR by only 2.6% and 3.6%, respectively, compared to the original algorithm. CONCLUSIONS: The addition of an NLP component to the existing algorithm had modest effect on WR (decrease of 1.4-12.5%), at the cost of sensitivity. For future implementation it will be a trade-off between optimal case-finding techniques versus practical considerations such as acceptability and availability of resources.

External validation of semi-automated surveillance algorithms for deep surgical site infections after colorectal surgery in an independent country.

BACKGROUND: Automated surveillance methods that re-use electronic health record data are considered an attractive alternative to traditional manual surveillance. However, surveillance algorithms need to be thoroughly validated before being implemented in a clinical setting. With semi-automated surveillance patients are classified as low or high probability of having developed infection, and only high probability patients subsequently undergo manual record review. The aim of this study was to externally validate two existing semi-automated surveillance algorithms for deep SSI after colorectal surgery, developed on Spanish and Dutch data, in a Swedish setting. METHODS: The algorithms were validated in 225 randomly selected surgeries from Karolinska University Hospital from the period January 1, 2015 until August 31, 2020. Both algorithms were based on (re)admission and discharge data, mortality, reoperations, radiology orders, and antibiotic prescriptions, while one additionally used microbiology cultures. SSI was based on ECDC definitions. Sensitivity, specificity, positive predictive value, negative predictive value, and workload reduction were assessed compared to manual surveillance. RESULTS: Both algorithms performed well, yet the algorithm not relying on microbiological culture data had highest sensitivity (97.6, 95%CI: 87.4-99.6), which was comparable to previously published results. The latter algorithm aligned best with clinical practice and would lead to 57% records less to review. CONCLUSIONS: The results highlight the importance of thorough validation before implementation in other clinical settings than in which algorithms were originally developed: the algorithm excluding microbiology cultures had highest sensitivity in this new setting and has the potential to support large-scale semi-automated surveillance of SSI after colorectal surgery.

Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms.

OBJECTIVE: Automated surveillance methods increasingly replace or support conventional (manual) surveillance; the latter is labor intensive and vulnerable to subjective interpretation. We sought to validate 2 previously developed semiautomated surveillance algorithms to identify deep surgical site infections (SSIs) in patients undergoing colorectal surgeries in Dutch hospitals. DESIGN: Multicenter retrospective cohort study. METHODS: From 4 hospitals, we selected colorectal surgery patients between 2018 and 2019 based on procedure codes, and we extracted routine care data from electronic health records. Per hospital, a classification model and a regression model were applied independently to classify patients into low- or high probability of having developed deep SSI. High-probability patients need manual SSI confirmation; low-probability records are classified as no deep SSI. Sensitivity, positive predictive value (PPV), and workload reduction were calculated compared to conventional surveillance. RESULTS: In total, 672 colorectal surgery patients were included, of whom 28 (4.1%) developed deep SSI. Both surveillance models achieved good performance. After adaptation to clinical practice, the classification model had 100% sensitivity and PPV ranged from 11.1% to 45.8% between hospitals. The regression model had 100% sensitivity and 9.0%-14.9% PPV. With both models, <25% of records needed review to confirm SSI. The regression model requires more complex data management skills, partly due to incomplete data. CONCLUSIONS: In this independent external validation, both surveillance models performed well. The classification model is preferred above the regression model because of source-data availability and less complex data-management requirements. The next step is implementation in infection prevention practices and workflow processes.

Reliability and validity of multicentre surveillance of surgical site infections after colorectal surgery.

BACKGROUND: Surveillance is the cornerstone of surgical site infection prevention programs. The validity of the data collection and awareness of vulnerability to inter-rater variation is crucial for correct interpretation and use of surveillance data. The aim of this study was to investigate the reliability and validity of surgical site infection (SSI) surveillance after colorectal surgery in the Netherlands. METHODS: In this multicentre prospective observational study, seven Dutch hospitals performed SSI surveillance after colorectal surgeries performed in 2018 and/or 2019. When executing the surveillance, a local case assessment was performed to calculate the overall percentage agreement between raters within hospitals. Additionally, two case-vignette assessments were performed to estimate intra-rater and inter-rater reliability by calculating a weighted Cohen's Kappa and Fleiss' Kappa coefficient. To estimate the validity, answers of the two case-vignettes questionnaires were compared with the answers of an external medical panel. RESULTS: 1111 colorectal surgeries were included in this study with an overall SSI incidence of 8.8% (n = 98). From the local case assessment it was estimated that the overall percent agreement between raters within a hospital was good (mean 95%, range 90-100%). The Cohen's Kappa estimated for the intra-rater reliability of case-vignette review varied from 0.73 to 1.00, indicating substantial to perfect agreement. The inter-rater reliability within hospitals showed more variation, with Kappa estimates ranging between 0.61 and 0.94. In total, 87.9% of the answers given by the raters were in accordance with the medical panel. CONCLUSIONS: This study showed that raters were consistent in their SSI-ascertainment (good reliability), but improvements can be made regarding the accuracy (moderate validity). Accuracy of surveillance may be improved by providing regular training, adapting definitions to reduce subjectivity, and by supporting surveillance through automation.

Information technology aspects of large-scale implementation of automated surveillance of healthcare-associated infections.

INTRODUCTION: Healthcare-associated infections (HAI) are a major public health concern. Monitoring of HAI rates, with feedback, is a core component of infection prevention and control programmes. Digitalization of healthcare data has created novel opportunities for automating the HAI surveillance process to varying degrees. However, methods are not standardized and vary widely between different healthcare facilities. Most current automated surveillance (AS) systems have been confined to local settings, and practical guidance on how to implement large-scale AS is needed. METHODS: This document was written by a task force formed in March 2019 within the PRAISE network (Providing a Roadmap for Automated Infection Surveillance in Europe), gathering experts in HAI surveillance from ten European countries. RESULTS: The document provides an overview of the key e-health aspects of implementing an AS system of HAI in a clinical environment to support both the infection prevention and control team and information technology (IT) departments. The focus is on understanding the basic principles of storage and structure of healthcare data, as well as the general organization of IT infrastructure in surveillance networks and participating healthcare facilities. The fundamentals of data standardization, interoperability and algorithms in relation to HAI surveillance are covered. Finally, technical aspects and practical examples of accessing, storing and sharing healthcare data within a HAI surveillance network, as well as maintenance and quality control of such a system, are discussed. CONCLUSIONS: With the guidance given in this document, along with the PRAISE roadmap and governance documents, readers will find comprehensive support to implement large-scale AS in a surveillance network.

Governance aspects of large-scale implementation of automated surveillance of healthcare-associated infections.

OBJECTIVES: Surveillance of healthcare-associated infections (HAI) is increasingly automated by applying algorithms to routine-care data stored in electronic health records. Hitherto, initiatives have mainly been confined to single healthcare facilities and research settings, leading to heterogeneity in design. The PRAISE network - Providing a Roadmap for Automated Infection Surveillance in Europe - designed a roadmap to provide guidance on how to move automated surveillance (AS) from the research setting to large-scale implementation. Supplementary to this roadmap, we here discuss the governance aspects of automated HAI surveillance within networks, aiming to support both the coordinating centres and participating healthcare facilities as they set up governance structures and to enhance involvement of legal specialists. METHODS: This article is based on PRAISE network discussions during two workshops. A taskforce was installed that further elaborated governance aspects for AS networks by reviewing documents and websites, consulting experts and organizing teleconferences. Finally, the article has been reviewed by an independent panel of international experts. RESULTS: Strict governance is indispensable in surveillance networks, especially when manual decisions are replaced by algorithms and electronically stored routine-care data are reused for the purpose of surveillance. For endorsement of AS networks, governance aspects specifically related to AS networks need to be addressed. Key considerations include enabling participation and inclusion, trust in the collection, use and quality of data (including data protection), accountability and transparency. CONCLUSIONS: This article on governance aspects can be used by coordinating centres and healthcare facilities participating in an AS network as a starting point to set up governance structures. Involvement of main stakeholders and legal specialists early in the development of an AS network is important for endorsement, inclusivity and compliance with the laws and regulations that apply.

PRAISE: providing a roadmap for automated infection surveillance in Europe.

INTRODUCTION: Healthcare-associated infections (HAI) are among the most common adverse events of medical care. Surveillance of HAI is a key component of successful infection prevention programmes. Conventional surveillance - manual chart review - is resource intensive and limited by concerns regarding interrater reliability. This has led to the development and use of automated surveillance (AS). Many AS systems are the product of in-house development efforts and heterogeneous in their design and methods. With this roadmap, the PRAISE network aims to provide guidance on how to move AS from the research setting to large-scale implementation, and how to ensure the delivery of surveillance data that are uniform and useful for improvement of quality of care. METHODS: The PRAISE network brings together 30 experts from ten European countries. This roadmap is based on the outcome of two workshops, teleconference meetings and review by an independent panel of international experts. RESULTS: This roadmap focuses on the surveillance of HAI within networks of healthcare facilities for the purpose of comparison, prevention and quality improvement initiatives. The roadmap does the following: discusses the selection of surveillance targets, different organizational and methodologic approaches and their advantages, disadvantages and risks; defines key performance requirements of AS systems and suggestions for their design; provides guidance on successful implementation and maintenance; and discusses areas of future research and training requirements for the infection prevention and related disciplines. The roadmap is supported by accompanying documents regarding the governance and information technology aspects of implementing AS. CONCLUSIONS: Large-scale implementation of AS requires guidance and coordination within and across surveillance networks. Transitions to large-scale AS entail redevelopment of surveillance methods and their interpretation, intensive dialogue with stakeholders and the investment of considerable resources. This roadmap can be used to guide future steps towards implementation, including designing solutions for AS and practical guidance checklists.

Validation of an algorithm for semiautomated surveillance to detect deep surgical site infections after primary total hip or knee arthroplasty-A multicenter study.

OBJECTIVE: Surveillance of healthcare-associated infections is often performed by manual chart review. Semiautomated surveillance may substantially reduce workload and subjective data interpretation. We assessed the validity of a previously published algorithm for semiautomated surveillance of deep surgical site infections (SSIs) after total hip arthroplasty (THA) or total knee arthroplasty (TKA) in Dutch hospitals. In addition, we explored the ability of a hospital to automatically select the patients under surveillance. DESIGN: Multicenter retrospective cohort study. METHODS: Hospitals identified patients who underwent THA or TKA either by procedure codes or by conventional surveillance. For these patients, routine care data regarding microbiology results, antibiotics, (re)admissions, and surgeries within 120 days following THA or TKA were extracted from electronic health records. Patient selection was compared with conventional surveillance and patients were retrospectively classified as low or high probability of having developed deep SSI by the algorithm. Sensitivity, positive predictive value (PPV), and workload reduction were calculated and compared to conventional surveillance. RESULTS: Of 9,554 extracted THA and TKA surgeries, 1,175 (12.3%) were revisions, and 8,378 primary surgeries remained for algorithm validation (95 deep SSIs, 1.1%). Sensitivity ranged from 93.6% to 100% and PPV ranged from 55.8% to 72.2%. Workload was reduced by ≥98%. Also, 2 SSIs (2.1%) missed by the algorithm were explained by flaws in data selection. CONCLUSIONS: This algorithm reliably detects patients with a high probability of having developed deep SSI after THA or TKA in Dutch hospitals. Our results provide essential information for successful implementation of semiautomated surveillance for deep SSIs after THA or TKA.

A framework to develop semiautomated surveillance of surgical site infections: An international multicenter study.

OBJECTIVE: Automated surveillance of healthcare-associated infections reduces workload and improves standardization, but it has not yet been adopted widely. In this study, we assessed the performance and feasibility of an easy implementable framework to develop algorithms for semiautomated surveillance of deep incisional and organ-space surgical site infections (SSIs) after orthopedic, cardiac, and colon surgeries. DESIGN: Retrospective cohort study in multiple countries. METHODS: European hospitals were recruited and selected based on the availability of manual SSI surveillance data from 2012 onward (reference standard) and on the ability to extract relevant data from electronic health records. A questionnaire on local manual surveillance and clinical practices was administered to participating hospitals, and the information collected was used to pre-emptively design semiautomated surveillance algorithms standardized for multiple hospitals and for center-specific application. Algorithm sensitivity, positive predictive value, and reduction of manual charts requiring review were calculated. Reasons for misclassification were explored using discrepancy analyses. RESULTS: The study included 3 hospitals, in the Netherlands, France, and Spain. Classification algorithms were developed to indicate procedures with a high probability of SSI. Components concerned microbiology, prolonged length of stay or readmission, and reinterventions. Antibiotics and radiology ordering were optional. In total, 4,770 orthopedic procedures, 5,047 cardiac procedures, and 3,906 colon procedures were analyzed. Across hospitals, standardized algorithm sensitivity ranged between 82% and 100% for orthopedic surgery, between 67% and 100% for cardiac surgery, and between 84% and 100% for colon surgery, with 72%-98% workload reduction. Center-specific algorithms had lower sensitivity. CONCLUSIONS: Using this framework, algorithms for semiautomated surveillance of SSI can be successfully developed. The high performance of standardized algorithms holds promise for large-scale standardization.

A diagnostic algorithm for the surveillance of deep surgical site infections after colorectal surgery.

OBJECTIVE: Surveillance of surgical site infections (SSIs) is important for infection control and is usually performed through retrospective manual chart review. The aim of this study was to develop an algorithm for the surveillance of deep SSIs based on clinical variables to enhance efficiency of surveillance. DESIGN: Retrospective cohort study (2012-2015). SETTING: A Dutch teaching hospital. PARTICIPANTS: We included all consecutive patients who underwent colorectal surgery excluding those with contaminated wounds at the time of surgery. All patients were evaluated for deep SSIs through manual chart review, using the Centers for Disease Control and Prevention (CDC) criteria as the reference standard. ANALYSIS: We used logistic regression modeling to identify predictors that contributed to the estimation of diagnostic probability. Bootstrapping was applied to increase generalizability, followed by assessment of statistical performance and clinical implications. RESULTS: In total, 1,606 patients were included, of whom 129 (8.0%) acquired a deep SSI. The final model included postoperative length of stay, wound class, readmission, reoperation, and 30-day mortality. The model achieved 68.7% specificity and 98.5% sensitivity and an area under the receiver operator characteristic (ROC) curve (AUC) of 0.950 (95% CI, 0.932-0.969). Positive and negative predictive values were 21.5% and 99.8%, respectively. Applying the algorithm resulted in a 63.4% reduction in the number of records requiring full manual review (from 1,606 to 590). CONCLUSIONS: This 5-parameter model identified 98.5% of patients with a deep SSI. The model can be used to develop semiautomatic surveillance of deep SSIs after colorectal surgery, which may further improve efficiency and quality of SSI surveillance.

Preoperative Oral Antibiotic Prophylaxis Reduces Surgical Site Infections After Elective Colorectal Surgery: Results From a Before-After Study.

BACKGROUND: Surgical site infections (SSIs) are common complications after colorectal procedures and remain an important source of morbidity and costs. Preoperative oral antibiotic prophylaxis is a potential infection control strategy, but its effectiveness without simultaneous use of mechanical bowel preparation (MBP) is unclear. In this study, we aimed to determine whether preoperative oral antibiotics reduce the risk of deep SSIs in elective colorectal surgery. METHODS: We performed a before-after analysis in a teaching hospital in the Netherlands. Patients who underwent surgery between January 2012 and December 2015 were included. On 1 January 2013, oral antibiotic prophylaxis with tobramycin and colistin was implemented as standard of care prior to colorectal surgery. The year before implementation was used as the control period. The primary outcome was a composite of deep SSI and/or mortality within 30 days after surgery. RESULTS: Of the 1410 patients, 352 underwent colorectal surgery in the control period and 1058 in the period after implementation of the antibiotic prophylaxis. We observed a decrease in incidence of the primary endpoint of 6.2% after prophylaxis implementation. When adjusted for confounders, the risk ratio for development of the primary outcome was 0.58 (95% confidence interval, 0.40-0.79). Other findings included a decreased risk of anastomotic leakage and a reduction in the length of postoperative stay. CONCLUSIONS: Preoperative oral antibiotic prophylaxis prior to colorectal surgery is associated with a significant decrease in SSI and/or mortality in a setting without MBP. Preoperative oral antibiotics can therefore be considered without MBP for patients who undergo colorectal surgery.

Designing Surveillance of Healthcare-Associated Infections in the Era of Automation and Reporting Mandates.

Surveillance and feedback of infection rates to clinicians and other stakeholders is a cornerstone of healthcare-associated infection (HAI) prevention programs. In addition, HAIs are increasingly included in public reporting and payment mandates. Conventional manual surveillance methods are resource intensive and lack standardization. Developments in information technology have propelled a movement toward the use of standardized and semiautomated methods.When developing automated surveillance systems, several strategies can be chosen with regard to the degree of automation and standardization and the definitions used. Yet, the advantages of highly standardized surveillance may come at the price of decreased clinical relevance and limited preventability. The choice among (automated) surveillance approaches, therefore, should be guided by the intended aim and scale of surveillance (eg, research, in-hospital quality improvement, national surveillance, or pay-for-performance mandates), as this choice dictates subsequent methods, important performance characteristics, and suitability of the data generated for the different applications.

Comparative gut microbiota and resistome profiling of intensive care patients receiving selective digestive tract decontamination and healthy subjects.

BACKGROUND: The gut microbiota is a reservoir of opportunistic pathogens that can cause life-threatening infections in critically ill patients during their stay in an intensive care unit (ICU). To suppress gut colonization with opportunistic pathogens, a prophylactic antibiotic regimen, termed "selective decontamination of the digestive tract" (SDD), is used in some countries where it improves clinical outcome in ICU patients. Yet, the impact of ICU hospitalization and SDD on the gut microbiota remains largely unknown. Here, we characterize the composition of the gut microbiota and its antimicrobial resistance genes ("the resistome") of ICU patients during SDD and of healthy subjects. RESULTS: From ten patients that were acutely admitted to the ICU, 30 fecal samples were collected during ICU stay. Additionally, feces were collected from five of these patients after transfer to a medium-care ward and cessation of SDD. Feces from ten healthy subjects were collected twice, with a 1-year interval. Gut microbiota and resistome composition were determined using 16S rRNA gene phylogenetic profiling and nanolitre-scale quantitative PCRs. The microbiota of the ICU patients differed from the microbiota of healthy subjects and was characterized by lower microbial diversity, decreased levels of Escherichia coli and of anaerobic Gram-positive, butyrate-producing bacteria of the Clostridium clusters IV and XIVa, and an increased abundance of Bacteroidetes and enterococci. Four resistance genes (aac(6')-Ii, ermC, qacA, tetQ), providing resistance to aminoglycosides, macrolides, disinfectants, and tetracyclines, respectively, were significantly more abundant among ICU patients than in healthy subjects, while a chloramphenicol resistance gene (catA) and a tetracycline resistance gene (tetW) were more abundant in healthy subjects. CONCLUSIONS: The gut microbiota of SDD-treated ICU patients deviated strongly from the gut microbiota of healthy subjects. The negative effects on the resistome were limited to selection for four resistance genes. While it was not possible to disentangle the effects of SDD from confounding variables in the patient cohort, our data suggest that the risks associated with ICU hospitalization and SDD on selection for antibiotic resistance are limited. However, we found evidence indicating that recolonization of the gut by antibiotic-resistant bacteria may occur upon ICU discharge and cessation of SDD.

Automated surveillance of healthcare-associated infections: state of the art.

PURPOSE OF REVIEW: This review describes recent advances in the field of automated surveillance of healthcare-associated infections (HAIs), with a focus on data sources and the development of semiautomated or fully automated algorithms. RECENT FINDINGS: The availability of high-quality data in electronic health records and a well-designed information technology (IT) infrastructure to access these data are indispensable for successful implementation of automated HAI surveillance. Previous studies have demonstrated that reliance on stand-alone administrative data is generally unsuited as sole case-finding strategy. Recent attempts to combine multiple administrative and clinical data sources in algorithms yielded more reliable results. Current surveillance practices are mostly limited to single healthcare facilities, but future linkage of multiple databases in a network may allow interfacility surveillance. Although prior surveillance algorithms were often straightforward decision trees based on structured data, recent studies have used a wide variety of techniques for case-finding, including logistic regression and various machine learning methods. In the future, natural language processing may enable the use of unstructured narrative data. SUMMARY: Developments in healthcare IT are rapidly changing the landscape of HAI surveillance. The electronic availability and incorporation of routine care data in surveillance algorithms enhances the reliability, efficiency and standardization of surveillance practices.

Semiautomated Surveillance of Deep Surgical Site Infections After Primary Total Hip or Knee Arthroplasty.

Manual surveillance of surgical site infections (SSIs) after total hip or knee arthroplasty is time-consuming and prone to error. Semiautomated surveillance based on routine care data extracted from electronic health records can retrospectively identify deep SSIs and substantially reduce workload while maintaining 100% sensitivity. Infect Control Hosp Epidemiol 2017;38:732-735.

Effect of isoniazid preventive therapy on tuberculosis or death in persons with HIV: a retrospective cohort study.

BACKGROUND: Isoniazid preventive therapy (IPT) is a recommended strategy for prevention of tuberculosis (TB) in persons with Human Immunodeficiency Virus (HIV) although the benefits have not been unequivocally demonstrated in routine clinical practice with widespread ART adoption. Therefore, we assessed the effectiveness of IPT in prevention of TB or death in patients treated with antiretroviral therapy (ART) in a chronic care setting. METHODS: Retrospective cohort study of HIV patients enrolled in chronic care from 2007 to 2013. Eligible participants were HIV infected subjects (age > 15 years) with no (history of) TB. The combined effect of IPT and ART on the composite outcome (TB or death) was estimated using time-dependent Cox regression with adjustment for baseline covariates. RESULTS: 1,922 patients were included, 374 (19.4 %) received IPT and 258 (13.4 %) developed TB or deceased. The median follow-up duration of the cohort was 839 days, with a total of 5491 person years. In unadjusted analysis, the combination of IPT and ART lowered the hazard of TB or death by 65 % [HR = 0.35; 95 % CI (0.16, 0.77)] compared to ART alone. Even after adjustment for confounders, the combined effect of ART and IPT resulted in a 60 % hazard reduction of TB or death in comparison to participants who received ART without IPT [HR = 0.40; 95 % CI (0.18, 0.87)]. The IPT-specific benefit in patients not receiving ART could not be reliably estimated due to high rates of ART adoption. CONCLUSION: The combined effect of IPT and ART to prevent TB or death in HIV patients in a non-experimental setting in comparison to ART alone was estimated to be 60 %.

Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review.

OBJECTIVE: Measuring the incidence of healthcare-associated infections (HAI) is of increasing importance in current healthcare delivery systems. Administrative data algorithms, including (combinations of) diagnosis codes, are commonly used to determine the occurrence of HAI, either to support within-hospital surveillance programmes or as free-standing quality indicators. We conducted a systematic review evaluating the diagnostic accuracy of administrative data for the detection of HAI. METHODS: Systematic search of Medline, Embase, CINAHL and Cochrane for relevant studies (1995-2013). Methodological quality assessment was performed using QUADAS-2 criteria; diagnostic accuracy estimates were stratified by HAI type and key study characteristics. RESULTS: 57 studies were included, the majority aiming to detect surgical site or bloodstream infections. Study designs were very diverse regarding the specification of their administrative data algorithm (code selections, follow-up) and definitions of HAI presence. One-third of studies had important methodological limitations including differential or incomplete HAI ascertainment or lack of blinding of assessors. Observed sensitivity and positive predictive values of administrative data algorithms for HAI detection were very heterogeneous and generally modest at best, both for within-hospital algorithms and for formal quality indicators; accuracy was particularly poor for the identification of device-associated HAI such as central line associated bloodstream infections. The large heterogeneity in study designs across the included studies precluded formal calculation of summary diagnostic accuracy estimates in most instances. CONCLUSIONS: Administrative data had limited and highly variable accuracy for the detection of HAI, and their judicious use for internal surveillance efforts and external quality assessment is recommended. If hospitals and policymakers choose to rely on administrative data for HAI surveillance, continued improvements to existing algorithms and their robust validation are imperative.